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This Study describes the procedures used to evaluate the effect of 28 
leadership development programs for young and middle-aged adults. The educational 
experiences were divided into 4 groups — low# medium# high# and very high -- 
depending on the level of the learners* participation and the extent to which the 
learning was focused on their interests and experiences. These factors were rated 
I on a subjective basis by the author. From a variety of approaches to predicting 

f successful group leaders, a series of short questions that actually discriminated 

successful from unsuccessful group leaders were selected, three attitude 
questionnaires were put together in a leadership inventory and studies of its 
reliability ranged from .70 to .90 (odd-even and test-retest). Studies of its validity In 
eight field situations ranged from .23 to .69 with ah average of about .50. These are 
multiple correlations and were established using the Aitken method with rank order 
correlations and the Jenkins method with product -moment correlations: The findings 
revealed that high participation led to a more significant attitude change than low 
participation, (nl) 
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There are three major areas that can be measured in an attempt to 
assess the impact of training or supervisory programs: 

1. changes in participants* on-the-job effectiveness. 

2. ' changes in knowledge, attitudes, and skills of participants. 

3. responses of participants to what they think they have learned. 

The most valid way of evaluating an educational program is to assess 
the change in behavior of people who participated in the program and 
perhaps compare it with people who did not participate. In practice it 
is usually very difficult to find accurate methods of measuring the 
appropriate behaviors, and to reduce the many other intervening variables 
accompanying the educational program. 

The knowledge, attitudes and skills of the participants can be j 

measured before and after a training program to see if they improve. \ 

However, a change in knowledge does not necessarily mean that there has 
been learning, if by learning we mean a change in behavior. Somehow the ! 

changes in knowledge, attitudes and skills have to be related to on-going | 

behavior. j 

The responses of participants about what they think they learned in 
an educational program is a useful source of information, especially in | 

planning a similar program. It is difficult for participants to be ! 

objective about what they learned and they often respond by describing 
their level of satisfaction with the educational experience. There is 
not necessarily any relationship betvreen satisfaction and learning, and 
spending many hours in a program may be a sufficient reason for feeling 
that something must have been learned. 



The Background of This Study 

Nine years ago I embarked on a project to try to find more satiOTactory 
procedures for selecting group leaders in community serving organizations. 
After trying many different approaches and techniques, a series of attitude 
questionnaires emerged as a very effective;': method for selecting successful 
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group leaders^. During these nine years this leadership research project 
has branched out in many different directions. One of the most interesting 
areas, and the one described in this report, was the use of the leadership 
inventory in assessing the impact of a variety of training and supervisory 
experiences that were designed to improve the effectiveness of group leaders. 

This study, then, describes the procedures used to evaluate the affect 
of twenty-eight leadership development programs for young and middle-aged 
adults. It is an example of procedure number two above - measuring changes 
in attitudes of participants - coupled with an established relationship 
between these attitudes and success on-the-job. 



Validation of the Attitude Scales 



The Dimock Leadership Inventory in its latest revision is composed of 
eighty-two questions answered on a five point Likert type scale, or by a 
true, uncertain or false response. The inventory is made up of the friendliness 
scale of the Guilford-Zimmerman Temperament Survey, a modification of the 
California F Scale (Adorno, et al , 1950) and questions made up by the author. 

It was designed to measure a flexible, cooperative, participative orientation 
to working with others. It has been validated on two hundred group leaders 
working in eight different settings. All the leaders completed the inventory 
in advance of their group work in an agency or camp and at the conclusion 
of their work their supervisors assessed their competence or the quality of 
their leadership. Correlations of the attitude scales with success on-the-job 
ranged from .23 to .69 in the different situations. Figure one below illustrates 
the predictive ability of the leadership inventory and establishes its 
usefulness as a measure of success on-the-job. 



Group 


Chances in 100 of being effective leaders. 
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Figure 1. 

A summary of 198 group leaders who worked in one of eight different 
situations shows that effective leaders^ can be selected 92 times 
out of 100 by not taking the 30% of applicants who scored lowest on 
these attitude scales. If the lowest 60% of applicants were rejected, 
effective leaders could be selected 97 times out of 100. 



1. Dlmock, H.G. , "Selecting and Training Group Leaders”, Montreal: Centre 

for Human Relations and Community Studies, Sir George Williams University, 
1968. 

Effective leaders are defined as people who did a passable job as assessed 
by their supervisors. 18% of the sample were assessed as ineffective 
leaders. 
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Participation in Leadership Development Pro grams 



In my research I have given my leadership inventory on a Kefo« 
and after basis to participants in a variety of leadership develop^nt 
situations. These have included many formal University courses, « 

residential training programs, -,ear 

job supervisory programs. As these studies accumulated it became clear 

that the level of participation in the programs was correlated "i^h the 
degree of attitude change. By participation I mean the extent to which 
the learners had a say in some of the decisions which 

practice this was assessed as the importance placed on their ideas and 
needs and the extent to which their experiences were used as the basis 

for learning. 



The previous section reported the close relationship between 
leadership inventory scores and on-the-job success. There does 
seem to be a test-retest affect using the Inventory. The tnree control 
groups to date show no change and one high participation group a 
was split with each half doing either the odd or even questions before 
and tL others after showed changes similar to other groups in comparable 
experiences. Thus change in attitudes as measured by the inventory can 
more safely be felt to be related to real learning. 



Affect of Participation on Learning 



The twenty-eight different educational experiences have been divided 
into four groups depending on the level of learners par icipa ion 
ihe experience%nd ?he extent to which the learning was f 3 

Interests and experiences. These factors were rated on a subjective bas 

by the author. 



Low Participation - 4 groups: 2 groups are University lecture 

courses in The social sciences (used as control groups), 

1 group is a control group for an inner city youth study 
project^; and 1 is a supervised work experience with very 

low participation. 



Medium Participation - 6 groups: all are supervised work experiences 

with some informal on-the-job training and include some 
participation. 



High Participation - 10 groups: eight groups are University 

courses in group development which involve learners in many 
of the decisions which affect them and through role-playing, 
demonstrations and films focus on many of their on-going 
experiences; 1 was a guidance group; and 1 was a supervised 
work experience involving high participation. 



1 . 



Dimock, B.G., "Group Guidance for Disadvantaged. Biner City Youth", 
Montreal: Centre for Human Relations and Community Studies, Sir 
George Williams University, 1968. 
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Very High Participation - 8 experience based learning groups. 
All were human relations training groups: 1 a University 

courset 3 executive training programs, 1 human relations 
for nurses, and 3 general lab. programs for heterogeneous 
group s . 

These twenty-eight groups were all assessed as to the amount of 
attitude change shown by the learners: no change (0); some change - 
modest change at the .05 level of significance (*); and considerable 
change - a great deal of change at the .01 level of significance 
(**). Figure 2 shows the results. 



Low Participation 

0 Political Science Class 

0 Economics Glass 

0 Rotary Camp Supervision 

0 Inner City Youth Control 



Medium Participation 

* Kanawana Camp Supervision *60 

** Group Advisors Supervision 

0 Group Supervisors Supervision 

0 Program Workers Supervision 

* Kanawana Camp Supervision *67 

* Howell Camp Supervision 



High Participation 

** Group Development Class *61 

** Group Development Class *62 

* Group Development Class *63 

* Group Development Class *64 

** Group Development Class *65 

* Group Development Glass *66 

* Group Development Class *67 

* Group Development Class *68 

** Sherbourne Camp Supervision 

0 Inner City Youth Training 



Very High Participation 

** Group Behavior Class 

** Executive Training *62 

* Executive Training *63 

** Executive Training *64 

* Nurses Training 

** Sask. H.R. Training 

* Teen H.R. Training 

0 Sir George H.R. Training 



Figure 2. The amount of learning under varying degrees of participation. 

0 = no change * = some change ** = considerable change 
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The affect of a participatory orientation to teaching adults on 
attitude change shows up quickly when six of ten groups show no change 
in low and medium participation situations and only two of eighteen groups 
in high and very high participation show no change. 



Review of Research Procedures 



From a variety of approaches to predicting successful group leaders, 
a series of short questions \tere selected that actually discriminated 
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succgss£u1 from unsuccGSs£ul group loadors* ThrG© attitudG qua st ionna ira s 
wara put togather in a leadership inventory and studies of its reliability 
ranged from .70 to .90 (odd-even and test retest). Studies of its validity 
in eight field situations ranged from .23 to .69 with an average of about 
.50. These are multiple correlations and were established using the Aitken 
method with rank order correlations and the Jenkins method with product- 
moment correlations. 

Test retest methods with control groups and a split half method 
with a training group were used to check the affect of re-doing the 
inventory. These control groups also checked the influence of "daily 
living" on the attitude scores. 

Most of the learning groups studied before and after their training 
or supervisory experience were less than thirty in number and a t test 
checking the significance of a difference between the means of two small 
correlated samples was used to assess the change. The results were 
interpreted using Fisher *s table of t with .05 considered as significant 
change and .01 as very significant change. 

Many of the groups involved in the research volunteered to participate 
and in all cases they were offered a report on their scores and a summary 
of the research to date. About three quarters of the participants did 
receive information on their scores and this process seemed to reduce or 
eliminate any resistance to the research. 



Hedley G. Dimock I 

January, 1969. I 
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